Data Deduplication With Random Substitutions

Data deduplication saves storage space by identifying and removing repeats in the data stream. Compared with traditional compression methods, schemes are more time efficient thus widely used large scale systems. In this paper, we provide an information-theoretic analysis on performance of algorithms streams which not exact. We introduce a source model probabilistic substitutions considered. More precisely, each symbol repeated string is substituted given edit probability. Deduplication both fixed-length scheme variable-length studied. The algorithm shown to be unsuitable for proposed as it does take into account Two modifications have performances within constant factor optimal knowledge parameters. also study conventional show that entropy becomes smaller, size compressed vanishes relative length uncompressed string, leading high ratios.